GH-50774: [CI][Python] Match Protobuf symbol visibility in bundled Substrait and ORC by tadeja · Pull Request #50792 · apache/arrow

tadeja · 2026-08-03T19:04:42Z

Rationale for this change

Fix #50774. After #50650, the Pyodide build failed while loading PyArrow with ImportError: dynamic module does not define module export function (PyInit_lib).
The ImportError is misleading because the PyInit_lib is present. The actual failure is an earlier abort during static initialization of libarrow_python.so, in Protobuf descriptor registration.

#50650 changed the order of libraries merged into libarrow_bundled_dependencies.a. The archive contains duplicate weak Protobuf/Abseil symbols emitted by the Protobuf runtime and generated code in Substrait and in ORC. These symbols had inconsistent visibility:
a) Protobuf was compiled with hidden visibility,
b) Substrait and ORC generated Protobuf code used default visibility,
so the order change caused wasm-ld to select symbols with different visibility.
(e.g. google::protobuf::internal::ZeroFieldsBase::Clear() changed from hidden in Protobuf's empty.pb.cc.o to default visible in Substrait's algebra.pb.cc.o)

The failure showed up in Emscripten, no failures on other platforms, but matching visibility globally is desirable to prevent any unintended exports and symbol interposition.

What changes are included in this PR?

Compile bundled Substrait and ORC with the same hidden visibility for symbols as Protobuf so not depending on archive order.
Also add debug for Emscripten CI builds --profiling-funcs.

Are these changes tested?

Yes, test-conda-python-emscripten passes
(Local repro, debug and fix verification with ARCH=amd64 archery docker run conda-python-emscripten, for symbols llvm-nm and wasm-ld)

Are there any user-facing changes?

No.

GitHub Issue: [CI][Python] Pyodide import error #50774

tadeja · 2026-08-03T19:05:27Z

@github-actions crossbow submit test-conda-python-emscripten

github-actions · 2026-08-03T19:06:55Z

⚠️ GitHub issue #50774 has been automatically assigned in GitHub to PR creator.

github-actions · 2026-08-03T19:09:17Z

Revision: f5714c0

Submitted crossbow builds: ursacomputing/crossbow @ actions-50e51eced4

Task	Status
test-conda-python-emscripten

tadeja · 2026-08-03T23:25:13Z

Although the first crossbow run failed with pre-existing script timeout at the end in the Chrome step, it actually shows problematic part now succeeds:
https://github.com/ursacomputing/crossbow/actions/runs/30844436419/job/91789163035#step:6:6200

> await pyodide.loadPackage('/build/python/dist/pyarrow-26.0.0.dev99-cp312-cp312-pyemscripten_2024_0_wasm32.whl');

Loading pyarrow
Loaded pyarrow

( While in the issue logs Failed to load pyarrow)

The log also shows the export is present in the wheel, meaning that the PyInit_lib ImportError is misleading message.

Locally I traced the cause (with fix reverted failure reproduced), symbolized by patching abort() in pyodide.asm.js to print the stack, relinked with --profiling-funcs

> await pyodide.loadPackage('/build/python/dist/pyarrow-26.0.0a0-cp312-cp312-pyemscripten_2024_0_wasm32.whl');
Loading pyarrow
ABORT:  Error
    at abort (/pyodide/pyodide.asm.js:10:6334)
    at _abort (/pyodide/pyodide.asm.js:10:289408)
    at invoke_v (/pyodide/pyodide.asm.js:10:1224993)
    at wasm://wasm/0267d6e2:wasm-function[13105]:0x531c4b
    at libarrow_python.so.google::protobuf::FileDescriptorProto::SharedDtor(google::protobuf::MessageLite&) (wasm://wasm/libarrow_python.so-0fa2ac3a:wasm-function[48283]:0x2517a8a)
    at invoke_vi (/pyodide/pyodide.asm.js:10:1223725)
    at libarrow_python.so.google::protobuf::FileDescriptorProto::~FileDescriptorProto() (wasm://wasm/libarrow_python.so-0fa2ac3a:wasm-function[48282]:0x2517795)
    at libarrow_python.so.google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) (wasm://wasm/libarrow_python.so-0fa2ac3a:wasm-function[48600]:0x2527cec)
    at invoke_iiii (/pyodide/pyodide.asm.js:10:1223129)
    at libarrow_python.so.google::protobuf::internal::AddDescriptors(google::protobuf::internal::DescriptorTable const*) (wasm://wasm/libarrow_python.so-0fa2ac3a:wasm-function[48861]:0x255572e)
Aborted()
Failed to load pyarrow
The following error occurred while loading pyarrow:
Aborted(). Build with -sASSERTIONS for more info.
[]

The abort happens during static initialization of libarrow_python.so, inside protobuf's descriptor registration.

tadeja · 2026-08-03T23:45:06Z

@github-actions crossbow submit test-conda-python-emscripten

github-actions · 2026-08-03T23:47:13Z

Revision: e44aaa2

Submitted crossbow builds: ursacomputing/crossbow @ actions-e7e0e9a62d

Task	Status
test-conda-python-emscripten

tadeja · 2026-08-04T11:05:55Z

@github-actions crossbow submit test-conda-python-emscripten r-recheck-most

github-actions · 2026-08-04T11:10:52Z

Revision: a7d0931

Submitted crossbow builds: ursacomputing/crossbow @ actions-c8b74835fc

Task	Status
r-recheck-most
test-conda-python-emscripten

tadeja · 2026-08-04T11:30:47Z

@github-actions crossbow submit test-conda-python-emscripten

github-actions · 2026-08-04T11:33:05Z

Revision: 8d456b3

Submitted crossbow builds: ursacomputing/crossbow @ actions-6f85329ef0

Task	Status
test-conda-python-emscripten

tadeja · 2026-08-04T20:49:25Z

@github-actions crossbow submit test-conda-python-emscripten

github-actions · 2026-08-04T20:52:20Z

Revision: 18130db

Submitted crossbow builds: ursacomputing/crossbow @ actions-24de2ba338

Task	Status
test-conda-python-emscripten

kou

Wow! Thanks!!!

kou · 2026-08-04T21:56:23Z


  add_library(substrait STATIC ${SUBSTRAIT_SOURCES})
  set_target_properties(substrait PROPERTIES POSITION_INDEPENDENT_CODE ON)
+  if(CMAKE_SYSTEM_NAME STREQUAL "Emscripten")


Can we remove this condition? (Can we always set CXX_VISIBILITY_PRESET and VISIBILITY_INLINES_HIDDEN?)

kou · 2026-08-04T22:00:56Z

BTW, can we always use --profiling-funcs for easy to debug? Does it have a large performance penalty? If so, can we enable it only in our CI?

kou · 2026-08-05T14:04:18Z

-        "-DZLIB_LIBRARY=$<TARGET_FILE:ZLIB::ZLIB>")
+        "-DZLIB_LIBRARY=$<TARGET_FILE:ZLIB::ZLIB>"
+        -DCMAKE_CXX_VISIBILITY_PRESET=hidden
+        -DCMAKE_VISIBILITY_INLINES_HIDDEN=ON)


Could you sort this argument list?

oh... those two lines are sorted now!

tadeja · 2026-08-05T14:41:19Z

@github-actions crossbow submit test-conda-python-emscripten

github-actions · 2026-08-05T14:44:43Z

Revision: a95ac9f

Submitted crossbow builds: ursacomputing/crossbow @ actions-578e2a8b50

Task	Status
test-conda-python-emscripten

tadeja · 2026-08-05T19:32:39Z

@github-actions crossbow submit test-ubuntu-22.04-cpp test-debian-13-cpp-amd64

github-actions · 2026-08-05T19:35:10Z

Revision: a95ac9f

Submitted crossbow builds: ursacomputing/crossbow @ actions-ee064bc184

Task	Status
test-debian-13-cpp-amd64
test-ubuntu-22.04-cpp

tadeja · 2026-08-05T21:34:27Z

Thanks @kou , this is now ready for review

kou

+1

conbench-apache-arrow · 2026-08-06T07:58:14Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit d6d8f03.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 30 possible false positives for unstable benchmarks that are known to sometimes produce them.

github-actions Bot added Component: C++ awaiting review Awaiting review labels Aug 3, 2026

tadeja added 3 commits August 4, 2026 13:03

Reorder for cross-compiled builds

86707e2

TEMP show symoblized stack - revert later

d0d997d

Update comment

a7d0931

tadeja force-pushed the 50774-emscr-import-pyinit branch from e44aaa2 to a7d0931 Compare August 4, 2026 11:04

Targeted reorder fix

8d456b3

tadeja requested a review from kou August 4, 2026 12:16

tadeja changed the title ~~GH-50774: [CI][Python] Reorder for cross-compiled builds~~ GH-50774: [CI][Python] Restore bundled archive order for Emscripten Aug 4, 2026

tadeja mentioned this pull request Aug 4, 2026

[R][CI] r-recheck-most test failing #50765

Closed

Alternative visibility fix

18130db

kou reviewed Aug 4, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 4, 2026

Set visibility always and add debug

49eb4ef

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 5, 2026

kou reviewed Aug 5, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 5, 2026

Lint and sort

a95ac9f

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 5, 2026

tadeja changed the title ~~GH-50774: [CI][Python] Restore bundled archive order for Emscripten~~ GH-50774: [CI][Python] Match Protobuf symbol visibility in bundled Substrait and ORC Aug 5, 2026

tadeja marked this pull request as ready for review August 5, 2026 21:32

tadeja requested review from assignUser, jonkeane and raulcd as code owners August 5, 2026 21:32

kou approved these changes Aug 6, 2026

View reviewed changes

kou merged commit d6d8f03 into apache:main Aug 6, 2026
69 of 70 checks passed

kou removed the awaiting change review Awaiting change review label Aug 6, 2026

github-actions Bot added the awaiting merge Awaiting merge label Aug 6, 2026

kou mentioned this pull request Aug 6, 2026

[CI][Python] Pyodide import error #50774

Closed

Uh oh!

Conversation

tadeja commented Aug 3, 2026 • edited by kou Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

tadeja commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

tadeja commented Aug 3, 2026

Uh oh!

tadeja commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

tadeja commented Aug 4, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026

Uh oh!

tadeja commented Aug 4, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026

Uh oh!

tadeja commented Aug 4, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

kou Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

kou commented Aug 4, 2026

Uh oh!

kou Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

tadeja Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

tadeja commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

Uh oh!

tadeja commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 5, 2026

Uh oh!

tadeja commented Aug 5, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

conbench-apache-arrow Bot commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tadeja commented Aug 3, 2026 •

edited by kou

Loading